Method, system and computer program product for providing driving assistance

ABSTRACT

A method for providing driving assistance is implemented by a driving assistance system and includes the steps of: continuously capturing images of the surrounding environment of the vehicle; performing an image processing procedure on at least one of the images; determining whether a traffic sign indicating a message is detected in the at least one of the images, and whether additional information indicating a condition associated with the message is detected with respect to the traffic sign; determining whether an alert action is to be executed based on the message and a current condition associated with the vehicle; and when the determination is affirmative, executing the alert action.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority of Taiwanese Invention PatentApplication No. 108134211, filed on Sep. 23, 2019.

FIELD

The disclosure relates to a method, a system and a computer programproduct for providing driving assistance.

BACKGROUND

Conventionally, traffic signs are regularly erected on roadsides and/orabove roads to provide various information or notification to road users(drivers, pedestrians, etc.). The traffic signs may be categorized intoa number of types; for example, Annex 1 of Vienna Convention on RoadSigns and Signals defines eight categories of signs. Different countriesmay adopt different types of traffic sign conventions.

It is noted that for some road users, reading the different types oftraffic signs may be difficult and distracting, and may potentiallyincrease the risks on the road for these road users.

SUMMARY

Therefore, one object of the disclosure is to provide a method forproviding driving assistance.

According to the disclosure, the method is implemented using a drivingassistance system that includes a processor and an image capturing unit.The method includes:

a) controlling, by the processor, the image capturing unit tocontinuously capture images of the surrounding environment of thevehicle;

b) performing, by the processor, an image processing procedure on atleast one of the images captured by the image capturing unit;

c) determining, by the processor, whether a traffic sign indicating thata message is detected in the at least one of the images;

d) when the determination in step c) is affirmative, determining, by theprocessor, whether additional information indicating a conditionassociated with the message is detected with respect to the trafficsign;

e) determining, by the processor, whether an alert action is to beexecuted based on at least the message indicated by the traffic sign anda current condition associated with the vehicle, the current conditionincluding one of a current time, a current weather, a movement of thevehicle, and any combination thereof; and

f) when the determination in step e) is affirmative, executing, by theprocessor, the alert action.

Another object of the disclosure is to provide a system that is capableof implementing the above-mentioned method.

According to one embodiment of the disclosure, the driving assistancesystem includes a processor and an image capturing unit connected to theprocessor. The processor is programmed to:

control the image capturing unit to continuously capture images of thesurrounding environment of the vehicle;

perform an image processing procedure on at least one of the imagescaptured by the image capturing unit;

determine whether a traffic sign indicating a message is detected in theat least one of the images;

when the determination is affirmative, determine whether additionalinformation indicating a condition associated with the message isdetected with respect to the traffic sign;

determine whether an alert action is to be executed based on at leastthe message indicated by the traffic sign and a current conditionassociated with the vehicle, the current condition including one of acurrent time, a current weather, a movement of the vehicle, and anycombination thereof; and

when the determination is affirmative, execute the alert action.

Another object of the disclosure is to provide a computer programproduct that is capable of implementing the above-mentioned method.

According to one embodiment of the disclosure, the computer programproduct includes instructions that, when executed by a processor of anin-vehicle driving assistance system, cause the processor to performsteps of the above-mentioned method.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and advantages of the disclosure will become apparent inthe following detailed description of the embodiments with reference tothe accompanying drawings, of which:

FIG. 1 is a block diagram illustrating a driving assistance systemaccording to one embodiment of the disclosure;

FIG. 2 is a flow chart illustrating steps of a method for providingdriving assistance according to one embodiment of the disclosure;

FIG. 3 is a schematic diagram of an exemplary traffic sign;

FIG. 4 is a flow chart illustrating steps of a method for providingdriving assistance according to one embodiment of the disclosure; and

FIG. 5 is a schematic diagram of an exemplary set of traffic signs.

DETAILED DESCRIPTION

Before the disclosure is described in greater detail, it should be notedthat where considered appropriate, reference numerals or terminalportions of reference numerals have been repeated among the figures toindicate corresponding or analogous elements, which may optionally havesimilar characteristics.

Throughout the disclosure, the term “connected to” may refer to a directconnection among a plurality of electrical apparatus/devices/equipmentsvia an electrically conductive material (e.g., an electrical wire), anindirect connection between two electrical apparatus/devices/equipmentsvia another one or more apparatus/devices/equipments, or wirelesscommunication.

FIG. 1 is a block diagram illustrating a driving assistance system 1according to one embodiment of the disclosure. In this embodiment, thedriving assistance system 1 is disposed in a vehicle (not depicted inthe drawings), and includes an image capturing unit 11, a data storage12, an output unit 13, a communication unit 15, and a processor 14 thatis electrically connected to the image capturing unit 11, the datastorage 12, the output unit 13, and the communication unit 15.

The image capturing unit 11 may be embodied using a driving videorecorder (DVR) with a wide angle lens that is capable of covering anangle of view of at least 120 degrees, and may be disposed in thevehicle to face outside of the vehicle for capturing images ofsurrounding environment of the vehicle in directions outwardly of thevehicle. In some embodiments, the image capturing unit 11 may include aplurality of DVRs facing different directions, and therefore may becapable of covering the surrounding environment of the vehicle in alldirections. In some embodiments, the image capturing unit 11 may beembodied using other types of digital image capturing devices that arecapable of capturing images, such as a still camera with a continuousshooting mode (also known as a burst mode) function to rapidly capture aplurality of images in succession.

The data storage 12 may be embodied using flash memory, a hard disk, asolid state disk (SSD) or other types of non-transitory storage medium.The data storage 12 stores a software application, a neural networkmodel for dynamic image recognition, and an optical characterrecognition (OCR) model. The neural network model for dynamic imagerecognition may be embodied using a convolutional neural network (CNN).

The software application includes instructions that, when executed bythe processor 14, cause the processor 14 to perform a number offunctions as described in the succeeding paragraphs. In someembodiments, the content stored in the data storage 12 (i.e., thesoftware application, the neural network model for dynamic imagerecognition, and the OCR model) may be packed in the form of a computerprogram product. The dynamic image recognition includes objectidentification.

In use, the processor 14 executing the software application may beconfigured to identify a traffic sign in an image, and to detect one ormore characters (e.g., word-characters (e.g., Chinese characters),letters, numbers, symbols, etc.) in the traffic sign and/or located inproximity to the traffic sign. Moreover, the processor 14 may beconfigured to determine a category of the traffic sign based on, forexample, a shape and one or more colors of the traffic sign, thecharacters in the traffic sign and/or located in proximity to thetraffic sign, etc.

It is noted that the detection of the traffic signs and the charactersusing the neural network model for dynamic image recognition and the OCRmodel are readily known in the related technical fields, and the detailsthereof are omitted herein for the sake of brevity.

The output unit 13 may be embodied using a display screen, an audiooutput component, or a combination thereof.

The processor 14 may include, but not limited to, a single coreprocessor, a multi-core processor, a dual-core mobile processor, amicroprocessor, a microcontroller, a digital signal processor (DSP), afield-programmable gate array (FPGA), an application specific integratedcircuit (ASIC), and/or a system on a chip (SoC), etc. In thisembodiment, the data storage 12 and the processor 14 may be integratedin the form of an in-vehicle computer, or a remote server that is insignal connection with the image capturing unit 11 and the output unit13.

The communication unit 15 may include at least one of a radio-frequencyintegrated circuit (RFIC), a short-range wireless communication modulesupporting a short-range wireless communication network using a wirelesstechnology of Bluetooth® and/or Wi-Fi, etc., or a mobile communicationmodule supporting telecommunication using Long-Term Evolution (LTE), thethird generation (3G) and/or fifth generation (5G) of wireless mobiletelecommunications technology, and/or the like.

In some embodiments, the driving assistance system 1 may be embodiedusing an electronic device (e.g., a smartphone, a dashboard camera,etc.).

FIG. 2 is a flow chart illustrating steps of a method for providingdriving assistance according to one embodiment of the disclosure. Inthis embodiment, the method is implemented by the driving assistancesystem 1 as described in FIG. 1 .

In use, when the user of the vehicle (e.g., a driver) is driving thevehicle on a road, he/she may operate the driving assistance system 1 toexecute the software application. In response, the processor 14 isprogrammed to control the components of the driving assistance system 1to perform the steps as described below.

In step S11, the processor 14 controls the image capturing unit 11 tocontinuously capture images of the surrounding environment of thevehicle. It is noted that in some embodiments, the image capturing unit11 may be controlled to continuously capture images regardless of thestate of motion of the vehicle (e.g., in motion, or idling).

In step S12, the processor 14 performs an image processing procedure onat least one of the images captured by the image capturing unit 11.Specifically, the processor 14 may obtain the images captured by theimage capturing unit 11 (the images may be extracted from a videorecorded by the DVR and stored in the data storage 12), and proceed toperform dynamic image recognition on the images to identify objects inthe images using the CNN model.

In this embodiment, the image capturing unit 11 is capable of capturingimages of the surrounding environment of the vehicle in variousdirections, covering a left-front side of the vehicle, a front side ofthe vehicle, and a right-front side of the vehicle. In embodiments, thedynamic image recognition may be performed with respect to variousportions of the captured images. For example, in this embodiment, thedynamic image recognition may be performed with respect to images of thesurrounding environment at one lateral side of the vehicle that is theside of a front-passenger seat (hereinafter referred to as “passengerside”). It is noted that since in different geological regions of theworld, the front-passenger seat may be located on different lateralsides in the vehicle, the dynamic image recognition may be performedwith respect to different portions of the images depending on thegeological region in which this method is performed.

In embodiments, when the vehicle is being driven on a road, one or moretraffic signs may be located in the front side or the lateral sides ofthe vehicle. As such, in step S13, the processor 14 determines whetherat least one traffic sign is detected in one of the images, that is,whether the objects identified in step S12 include a traffic sign. Atraffic sign may be in the form of a plate, a light device, a displayscreen or the like, and may contain various types of information,instructions, rules, restrictions, or requirements, etc. In thisembodiment, the traffic sign may be restrictive in nature, i.e., it mayindicate a message such as a speed limit, a prohibited action, etc. Inthis embodiment, the processor 14 may perform step S13 by determiningwhether a flat surface is detected in the images. The presence of a flatsurface may indicate that a traffic sign is detected. In otherembodiments, various ways that are known in the related art fordetecting traffic signs may be employed. After it is determined that atraffic sign is detected, the processor 14 may further identify thecontent of the traffic sign (i.e., determine the message) using the OCRmodel or other recognition techniques that are commercially available.In one example, the type of a traffic sign may be determined using ashape and/or a color of the traffic sign (e.g., in Taiwan, a maximumspeed limit traffic sign may be in a circular shape with a red outerring, and a minimum speed limit traffic sign may be in a circular shapewith a blue background). After detecting the shape and/or a color of thetraffic sign, the processor 14 may further detect the characters on thetraffic sign to determine the details included in the traffic sign. Forexample, the characters “70” on a traffic sign in a circular shape witha red outer ring may indicate that a 70 kilometers per hour maximumspeed limit is in effect.

It is noted that, in some embodiments, the processor 14 may beconfigured to perform the dynamic image recognition with respect to aportion of the image. For example, for one image captured by the imagecapturing unit 11, the processor 14 may be configured to partition theimage into three portions (e.g., a top portion, a middle portion and abottom portion) and perform the dynamic image recognition with respectto the middle portion of the image. In this manner, the operation ofdetecting a traffic sign may be done more efficiently as the amount ofdata needed to be processed by the processor 14 is significantlyreduced.

When the determination of step S13 is affirmative, the flow proceeds tostep S14. Otherwise, the flow goes back to step S13 periodically todetermine whether at least one traffic sign is detected in one of theimages subsequently captured by the image capturing unit 11. Forexample, in this embodiment, when no traffic sign is detected, theprocessor 14 may perform the detection again after a predetermined timeperiod (e.g., 500 milliseconds).

In step S14, the processor 14 determines whether additional informationindicating a condition associated with the message is detected inrelation to the traffic sign (e.g., the conditions may be in the trafficsign itself or in proximity to the traffic sign). Essentially, step S14is to determine whether the traffic sign is a conditional traffic signmeaning that the restriction or requirement or information of theconditional traffic sign only applies if certain condition as describedin the traffic sign (or proximate to the traffic sign) is met (e.g., therestriction/requirement/information only applies to a specific type ofvehicle, e.g., buses, or the restriction/requirement/information onlyapplies during certain time period(s) of the day (from 6 a.m. to 8p.m.).

Specifically, in this embodiment, the processor 14 is first configuredto define a detection range in said one of the images that is associatedwith the traffic sign. In this embodiment, the detection range may be ina form of a circle with a center that coincides with a geometric centerof the traffic sign in said one of the images. A size of the detectionrange is positively related to the size of the traffic sign in said oneof the images. For example, the size of the detection range may be twotimes the size of the traffic sign in said one of the images.

Then, the processor 14 determines whether additional information isdetected within the detection range. When the determination of step S14is negative, the flow proceeds to step S15. Otherwise, the flow proceedsto step S16. It is noted that, in this embodiment, the processor 14 isnot required to perform the detection with respect to the entirety ofthe images captured by the image capturing unit 11. In this manner, theoperation of determining whether additional information is detectedwithin the detection range may be done more efficiently as the amount ofdata needed to be processed by the processor 14 is reduced.

In step S15, the processor 14 determines whether an alert action is tobe executed based on at least the message indicated by the traffic signand a current condition associated with the vehicle. It is noted thatthe current condition may include one or more of a current time, acurrent weather, and a movement of the vehicle (e.g., a speed, a turningdirection, a lane on which the vehicle is moving, etc.). When thedetermination of step S15 is affirmative, the processor 14 executes thealert action. In this embodiment, the processor 14 controls the outputunit 13 of the driving assistance system 1 to output an alert (such asan alerting text, a warning sound, a speech that reads the alerting textout loud, etc.) associated with the message. Otherwise, the processor 14may deem that no action is required, and the method is terminated. Inother embodiments, the method may be implemented again after apredetermined time period (e.g., 500 milliseconds).

In step S16, the processor 14 identifies the condition included in theadditional information, and determines whether at least part of thecurrent condition fits at least part of the condition. In thisembodiment, the processor 14 identifies the condition included in theadditional information using the OCR model or other recognitiontechniques that are commercially available. When it is determined thatat least part of the current condition fits at least part of thecondition, the flow proceeds to step S17. Otherwise, the processor 14may deem that no action is required, and the method is terminated. Inother embodiments, the method may be implemented again after apredetermined time period (e.g., 500 milliseconds).

In step S17, the processor 14 determines whether an alert action is tobe executed based on at least the message indicated by the traffic signand the current condition associated with the vehicle.

Specifically, the operations of step S17 include that the processor 14determines whether at least part of the current condition violates atleast part of the message. When it is determined that at least part ofthe current condition violates at least part of the message, theprocessor 14 executes the alert action. In this embodiment, theprocessor 14 controls the output unit 13 of the driving assistancesystem 1 to output an alert (such as a text, a warning sound, etc.)associated with the message. Otherwise, when it is determined that nopart of the current condition violates the message, the processor 14 mayperform the determination again after a predetermined time period (e.g.,one second), or to initiate the method again.

In use, when the vehicle is moving, a traffic sign 2 as shown in FIG. 3may be detected by the image capturing unit 11. The traffic sign 2 mayinclude a graphic part 21 that indicates a message (e.g., “left turn isprohibited”), and a text part 22 that indicates a condition associatedwith the message (e.g., the specified time periods 7:00 to 9:00 AM and3:30 to 5:30 PM, during which left turn is prohibited).

As such, the processor 14 determines in step S13 that a traffic sign isdetected, determines the message (i.e., left turn is prohibited), andthe flow proceeds to step 14 to determine whether additional informationindicating a condition associated with the message is detected in or inproximity to the traffic sign 2. When the text part 22 is detected, theflow goes to step S16.

In step S16, the processor 14 identifies the condition included in theadditional information. Then, the processor 14 determines whether thecurrent time is within a specified time period defined in the condition(i.e., 7:00 to 9:00 AM or 3:30 to 5:30 PM). When the determination isnegative, it means that left turn is allowed at the current time, andthe method may be terminated. When the determination is affirmative, theflow proceeds to step S17 to determine whether a movement of the vehicleindicates that the vehicle is undertaking the prohibited action (leftturn), and to prompt execution of the alert action when the movement ofthe vehicle violates the message (i.e., the vehicle is making a leftturn). In this embodiment, the determination may be performed by lookingup various parameters from different components of the vehicle, such asa steering wheel being turned counter-clockwise by, for example, morethan a preset angle, a left turn directional light being activated, apre-determined navigation route indicating a left turn is imminent, etc.

In other embodiments, different kinds of traffic signs may be detected.For example, when a speed limit traffic sign is detected, the messagemay be a maximum speed limit and/or a minimum speed limit, and thecondition may be a classification of the vehicle (for example, thetraffic sign may indicate speed limits imposed to light vehicles). Inthis case, the processor may first determine the classification of thevehicle (e.g., using original equipment manufacturer (OEM) data storedin the data storage 12) to determine whether the vehicle is classifiedas a light vehicle. When it is determined that the vehicle is classifiedas a light vehicle, the processor 14 may determine, based on aspeedometer of the vehicle, whether the vehicle is moving faster thanthe maximum speed limit or slower than the minimum speed limit, and toprompt execution of the alert action when the movement of the vehicleviolates at least part of the message (is moving faster than the maximumspeed limit or slower than the minimum speed limit).

In another example, the condition may be a weather condition of thesurrounding environment of the vehicle (for example, the traffic signmay indicate a maximum speed limit imposed in case of rain, fog, etc.).In this case, the processor 14 may first determine a current weathercondition (e.g., controlling the communication unit 15 to communicatewith a remote server to obtain the current weather condition) todetermine whether the current weather condition is the weather conditionspecified by the additional information related to the traffic sign.When it is determined that the current weather condition meets theweather condition specified by the additional information related to thetraffic sign, the processor 14 may determine, based on a speedometer ofthe vehicle, whether the vehicle is moving faster than the maximum speedlimit, and to prompt execution of the alert action when the movement ofthe vehicle violates the message (i.e., is moving faster than themaximum speed limit).

In some examples, the message may be a size limit (e.g., a height, awidth, etc.) and a weight limit. In such cases, the processor 14 mayaccess the OEM data stored in the data storage 12 to determine whetherthe vehicle is within the size limit specified by the traffic sign.

Using the above method, the driving assistance system 1 may beconfigured to determine a message of a traffic sign detected while thevehicle is moving, and to determine a condition associated with messageof the traffic sign. Afterward, the driving assistance system 1 may beconfigured to determine whether the current condition of the vehicleindicates that the vehicle may be performing a prohibited action, and toperform an alert action accordingly to notify the driver.

FIG. 4 is a flow chart illustrating steps of a method for providingdriving assistance according to one embodiment of the disclosure. Inthis embodiment, the method is implemented by the driving assistancesystem 1 as depicted in FIG. 1 . It is noted that the steps of themethod in this embodiment deal with the potential case that multipletraffic signs are detected in a same image.

In step S21, the processor 14 controls the image capturing unit 11 tocontinuously capture images of the surrounding environment of thevehicle.

In step S22, the processor 14 performs an image processing procedure onat least one of the images captured by the image capturing unit 11.Specifically, the processor 14 performs dynamic image recognition on atleast one of the images to identify objects in the image(s) using theCNN model.

In step S23, the processor 14 determines whether a first traffic signindicating a first message is detected. After it is determined that afirst traffic sign is detected, the processor 14 may further identifythe content of the first traffic sign (i.e., determine the message)using the OCR model or other recognition techniques that arecommercially available. When the determination of step S23 isaffirmative, the flow proceeds to step S24. Otherwise, the flow goesback to step S23 periodically to determine whether at least one trafficsign is detected in one of the images subsequently captured by the imagecapturing unit 11. For example, in this embodiment, when no traffic signis detected, the processor 14 may perform the detection again after apredetermined time period (e.g., 500 milliseconds).

In step S24, the processor 14 determines whether additional informationindicating a condition associated with the message is detected in or inproximity to the first traffic sign, and whether a second traffic signindicating a second message is detected. Based on the determination ofstep S24, one of the steps S25 to S27 as described below may beimplemented. In one example, the first traffic sign may indicate amaximum speed limit (i.e., the first message) with a condition, and thesecond traffic sign may indicate another maximum speed limit (i.e., thesecond message).

Specifically, in this embodiment, the processor 14 is first configuredto define a detection range in said one of the images that is associatedwith the first traffic sign. In this embodiment, the detection range maybe in a form of a circle with a center that coincides with a geometriccenter of the first traffic sign in the image. A size of the detectionrange is positively related to the size of the first traffic sign in theimage. For example, the size of the detection range may be two times thesize of the first traffic sign in the image. Then, the processor 14determines whether the additional information and the second trafficsign are detected within the detection range. In the case that noadditional information and no additional traffic sign is detected, theflow proceeds to step S25, in which the processor 14 determines whetheran alert action is to be executed based on at least the messageindicated by the traffic sign and a current condition associated withthe vehicle. It is noted that in this case, the operations of step S25may be done by the processor 14 in a manner similar to that of step S15(see FIG. 2 ).

In the case that additional information is detected and no additionaltraffic sign is detected, the flow proceeds to step S26, in which theprocessor 14 determines whether at least part of the current conditionfits at least part of the condition. It is noted that in this case, theoperations of step S26 may be done by the processor 14 in a mannersimilar to that of steps S16 and S17 (see FIG. 2 ).

In the case that both the additional information and the second trafficsign are detected, the flow proceeds to steps S27 and S28, in which theprocessor 14 selects one of the first traffic sign and the secondtraffic sign as a to-be-dealt-with traffic sign.

Specifically, the detected traffic signs may be in the form of a set oftraffic signs 3 as shown in FIG. 5 . The set of traffic signs 3 includesa first traffic sign 31A, a second traffic sign 31B, and an image 32.The first traffic sign 31A and the second traffic sign 31B may beassociated with a first message (e.g., a maximum speed limit or aminimum speed limit) and a second message (e.g., a different maximumspeed limit or a different minimum speed limit), respectively. The image32 may include graphic information that constitutes a condition (e.g., aspecific weather condition) and may be associated with one of the firsttraffic sign 31A and the second traffic sign 31B. In other embodiments,the image 32 may contain characters (such as texts and numbers) thatprovide additional information.

It is noted that in this example, each of the first traffic sign 31A,the second traffic sign 31B and the image 32 is enclosed by broken lines(which may not actually be present on the traffic signs).

In this embodiment, the processor 14 selects one of the traffic signsthat comes with the additional information (i.e., the first traffic sign31A) as a primary traffic sign in step S27. Afterward, the flow proceedsto step S28, in which the processor 14 determines whether at least partof the current condition fits at least part of the condition. Forexample, in the case where the set of traffic signs 3 is detected andthe condition indicates a raining weather, the processor 14 may access aremote server to obtain the current weather condition to determinewhether the current weather condition is the weather condition specifiedby the image 32 (i.e., raining weather).

When it is determined that the current weather condition is the weathercondition specified by the image 32, the flow proceeds to step S29, inwhich the processor 14 may determine, based on a speedometer of thevehicle, whether the vehicle is moving faster than the maximum speedlimit indicated by the first traffic sign 31A (110 kilometers per hour),and prompt execution of the alert action when the movement of thevehicle violates the first message (the vehicle is moving at a speedexceeding 110 kilometers per hour). In other words, when thedetermination of step S28 is affirmative, then step S29 is performedwith respect to the primary traffic sign which serves as theto-be-dealt-with traffic sign.

On the other hand, when it is determined in step S28 that the currentweather condition is not the weather condition specified by the image32, the flow proceeds to step S30, in which the processor 14 maydetermine, based on a speedometer of the vehicle, whether the vehicle ismoving faster than the maximum speed limit indicated by the secondtraffic sign 31B (130 kilometers per hour), and prompt the execution ofthe alert action when the movement of the vehicle violates the secondmessage (the vehicle is moving at a speed exceeding 130 kilometers perhour). In other words, when the determination of step S28 is negative,then step S30 is performed with respect to the second traffic sign whichserves as the to-be-dealt-with traffic sign.

In some examples, in processing the detected set of the traffic signs 3,when it is determined that the messages of the first traffic sign 31Aand the second traffic sign 31B are of the same kind (e.g., both aremaximum speed limits), and the condition typically indicates that therelevant message would be more restrictive (e.g., typically, a lowermaximum speed limit applies for a raining weather or foggy weather thana good (normal) weather), the processor 14 may directly select thetraffic sign that is associated with the more restrictive message (e.g.,a lower one of the maximum speed limits, which may be 110 kilometers perhour) as the primary traffic sign in step S27, and to subsequentlyperform steps S28 to S30 accordingly.

It is noted that while in this embodiment, the first traffic sign 31Aand the second traffic sign 31B may not be detected in the same timeinstance, the processor 14 is still capable of correctly selecting oneof the first traffic sign 31A and the second traffic sign 31B as theprimary traffic sign based on the condition, and of using one of thetraffic signs that is applicable to the current condition of the vehicleto determine whether to perform the alert action. That is to say, undernormal (e.g., not raining and not foggy) weather conditions, theprocessor 14 is programmed to determine whether the vehicle is movingfaster than 130 kilometers per hour, and on a rainy day, the processor14 is programmed to determine whether the vehicle is moving faster than110 kilometers per hour.

In one embodiment, the operations of step S27, in which the processor 14selects one of the first traffic sign and the second traffic sign as theprimary traffic sign, may be performed in a different manner from thatpreviously described.

Specifically, in such embodiment, the processor 14 determines whetherone of the traffic signs is enclosed together with the additionalinformation by a visible border and the other traffic sign is notenclosed within the visible border. In such a case, the processor 14 mayselect the one of the traffic signs that is enclosed together with theadditional information by a visible border as the primary traffic sign.Using the set of traffic signs 3 as shown in FIG. 5 as an example, whena visible border 33B that encloses the first traffic sign 31A and theimage 32, and a visible border 33C that encloses the second traffic sign31B are detected, the processor 14 may select the first traffic sign31A, which is assumed to be associated with the image 32 (and thereforethe additional information), as the primary traffic sign.

According to one embodiment of the disclosure, there is provided acomputer program product including instructions that, when executed by aprocessor of an in-vehicle driving assistance system, cause theprocessor to perform steps of a method for providing driving assistance,as described in any of the above embodiments. The computer programproduct may be stored in a non-transitory computer readable storagemedium (such as a flash drive, a compact disc, a cloud drive, etc.).

According to some embodiments, the driving assistance system 1 may beconfigured to obtain the current weather via one of a number of wayswithout communicating with the external servers. For example, theprocessor 14 may be programmed to determine the current weather byexecuting the image processing procedure, which may be programmed todetermine the current weather based on the images captured by the imagecapturing unit 11. Alternatively, the processor 14 may be programmed todetermine the current weather by detecting states of some components ofthe vehicle, such as whether a windscreen wiper is activated, whether awindshield has raindrops thereon, etc. It is noted that thedetermination of the current weather based on the images may beavailable in the related art, so details thereof are omitted herein forthe sake of brevity.

In another case, the determination in step S28 may be initially madebased on the current weather obtained from the remote server. When it isdetermined that the current weather fits the condition (i.e., israining, snowing, foggy, etc.), the flow proceeds to step S29. On theother hand, when it is determined that the current weather does not fitthe condition (i.e., is sunny, cloudy, etc.), the processor 14 mayfurther determine the current weather based on the states of somecomponents of the vehicle and/or the result of the image processingprocedure. When the further determination also indicates that thecurrent weather does not fit the condition, the flow proceeds to stepS30. On the other hand, when the further determination indicates thatthe current weather fits the condition, the flow proceeds to step S29.

In this manner, the flow of the above embodiment may serve to doublecheck on the information obtained from the remote server, so as toeliminate the potential situation that the current weather obtained fromthe remote server is not the same as the weather actually experienced bythe vehicle.

In those cases, the driver may manually switch to adopt the above mannerfor implementing the determination. In the above cases, some of theoperations in the method may be done more efficiently and costeffectively since the need for external input (e.g., the current weatherfrom the remote servers) may be eliminated, and may be implemented usingthe components that are already present in the vehicle (that is, noadditional component is needed to be installed in the vehicle).

To sum up, the embodiments of the disclosure provide a method, a systemand a computer program product that is capable of capturing images of atraffic sign on the road, identifying the message and the relevantcondition associated with the traffic sign, and determining whether thecurrent condition of the vehicle violates the message. When thedetermination is affirmative, an alert action is executed, which mayinclude outputting an alerting text, a warning sound, a speech thatreads the alerting text out loud, etc., for alerting the user of thevehicle. In this manner, the user may be more focused on driving itselfand less distracted by the need to look at the traffic signs.

In the description above, for the purposes of explanation, numerousspecific details have been set forth in order to provide a thoroughunderstanding of the embodiments. It will be apparent, however, to oneskilled in the art, that one or more other embodiments may be practicedwithout some of these specific details. It should also be appreciatedthat reference throughout this specification to “one embodiment,” “anembodiment,” an embodiment with an indication of an ordinal number andso forth means that a particular feature, structure, or characteristicmay be included in the practice of the disclosure. It should be furtherappreciated that in the description, various features are sometimesgrouped together in a single embodiment, figure, or description thereoffor the purpose of streamlining the disclosure and aiding in theunderstanding of various inventive aspects, and that one or morefeatures or specific details from one embodiment may be practicedtogether with one or more features or specific details from anotherembodiment, where appropriate, in the practice of the disclosure.

While the disclosure has been described in connection with what areconsidered the exemplary embodiments, it is understood that thisdisclosure is not limited to the disclosed embodiments but is intendedto cover various arrangements included within the spirit and scope ofthe broadest interpretation so as to encompass all such modificationsand equivalent arrangements.

What is claimed is:
 1. A method for providing driving assistance, themethod being implemented using a driving assistance system that includesa processor and an image capturing unit, and comprising steps of: a)controlling, by the processor, the image capturing unit to continuouslycapture images of surrounding environment of the vehicle; b) performing,by the processor, an image processing procedure on at least one of theimages captured by the image capturing unit: c) determining, by theprocessor, whether a traffic sign indicating that a message is detectedin the at least one of the images: d) when the determination in step c)is affirmative, determining, by the processor, whether additionalinformation indicating a condition associated with the message isdetected with respect to the traffic sign; e) determining, by theprocessor, whether an alert action is to be executed based on at leastthe message indicated by the traffic sign and a current conditionassociated with the vehicle, the current condition including one of acurrent time, a current weather, a movement of the vehicle, and anycombination thereof; and f) when the determination in step e) isaffirmative, executing, by the processor, the alert action; wherein stepe) includes determining whether at least part of the current conditionfits at least part of the condition, and when it is determined that atleast part of the current condition fits at least part of the condition,determining whether at least part of the current condition violates atleast part of the message, and determining an alert action is to beexecuted when at least part of the current condition violates at leastpart of the message; and wherein step f) includes controlling an outputunit of the driving assistance system to output an alert associated withthe message, further comprising, between steps d) and e): g) when aplurality of traffic signs are detected in the at least one of theimages in step c) and additional information associated with one of thetraffic signs is detected with respect to the one of the traffic signsin step d), selecting one of the traffic signs as a to-be-dealt-withtraffic sign; wherein step e) is performed with respect to theto-be-dealt-with traffic sign wherein step g) includes: when it isdetermined that one of the traffic signs is enclosed together with theadditional information within a visible border and other one of thetraffic signs is not enclosed within the visible border, performing stepe) with respect to the one of the traffic signs serving as theto-be-dealt-with traffic sign.
 2. The method of claim 1, wherein step d)includes: defining a detection range in the at least one of the imagesthat is associated with the traffic sign, a size of the detection rangebeing positively related to a size of the traffic sign in the at leastone of the images; and determining whether the additional information isdetected within the detection range.
 3. The method of claim 1, whereinstep g) includes: selecting one of the traffic signs with the additionalinformation as a primary traffic sign; determining whether at least partof the current condition fits at least part of the condition; when thedetermination is affirmative, performing step e) with respect to theprimary traffic sign serving as the to-be-dealt-with traffic sign; andwhen the determination is negative, performing step e) with respect tothe other traffic sign serving as the to-be-dealt-with traffic sign. 4.The method of claim 1, the messages for the traffic signs includingspeed limits, the additional information including a specific weathercondition, wherein step g) is performed by comparing the current weatherwith the specific weather condition, and using the one of the trafficsigns with the additional information as the to-be-dealt-with trafficsign when the current weather fits the specific weather condition.
 5. Adriving assistance system comprising a processor and an image capturingunit connected to said processor, said processor being programmed to:control said image capturing unit to continuously capture images ofsurrounding environment of the vehicle; perform an image processingprocedure on at least one of the images captured by said image capturingunit; determine whether a traffic sign indicating a message is detectedin the at least one of the images; when the determination isaffirmative, determine whether additional information indicating acondition associated with the message is detected with respect to thetraffic sign; determine whether an alert action is to be executed basedon at least the message indicated by the traffic sign and a currentcondition associated with the vehicle, the current condition includingone of a current time, a current weather, a movement of the vehicle, andany combination thereof; and when the determination is affirmative,execute the alert action; the driving assistance system furthercomprising an output unit, wherein: in determining whether an alertaction is to be executed, said processor is programmed to determinewhether at least part of the current condition fits at least part of thecondition, and when it is determined that at least part of the currentcondition fits at least part of the condition, to determine whether atleast part of the current condition violates at least part of themessage, and determine an alert action is to be executed when at leastpart of the current condition violates at least part of the message; andin executing the alert action, said processor is programmed to controlsaid output unit of the driving assistance system to output an alertassociated with the message, wherein said processor is furtherprogrammed to, prior to determining whether an alert action is to beexecuted: when a plurality of traffic signs are detected in the at leastone of the images and additional information associated with one of thetraffic signs is detected with respect to the one of the traffic signs,select one of the traffic signs as a to-be-dealt-with traffic sign;wherein determining whether an alert action is to be executed isperformed by said processor with respect to the to-be-dealt-with trafficsign.
 6. The driving assistance system of claim 5, wherein indetermining whether additional information is detected, said processoris programmed to: define a detection range in the at least one of theimages that is associated with the traffic sign, a size of the detectionrange being positively related to a size of the traffic sign in the atleast one of the images; and determine whether the additionalinformation is detected within the detection range.
 7. The drivingassistance system of claim 5, wherein in selecting one of the trafficsigns as a to-be-dealt-with traffic sign, said processor is furtherprogrammed to: select one of the traffic signs with the additionalinformation as a primary traffic sign; determine whether at least partof the current condition fits at least part of the condition; when thedetermination is affirmative, determine whether the alert action is tobe executed with respect to the primary traffic sign serving as theto-be-dealt-with traffic sign; and when the determination is negative,determine whether the alert action is to be executed with respect to theother traffic sign serving as the to-be-dealt-with traffic sign.
 8. Thedriving assistance system of claim 5, the messages for the traffic signsincluding speed limits, the additional information including a specificweather condition, wherein in selecting one of the traffic signs as ato-be-dealt-with traffic sign, said processor is programmed to comparethe current weather with the specific weather condition, and use the oneof the traffic signs with the additional information as theto-be-dealt-with traffic sign when the weather fits the specific weathercondition.
 9. The driving assistance system of claim 5, wherein inselecting one of the traffic signs as a to-be-dealt-with traffic sign,said processor is programmed to: when it is determined that one of thetraffic signs is enclosed together with the additional informationwithin a visible border and other one of the traffic signs is notenclosed within the visible border, determine whether an alert action isto be executed with respect to the one of the traffic signs serving asthe to-be-dealt-with traffic sign.